Efficient Global Optimization

The optimization method mentioned last post is an improvement on Efficient Global Optimization, presented in the paper Efficient Global Optimization of ExpensiveBlack-Box Functions (also available from on one of the author's websites)


Another paper listed in the references, A Taxonomy of Global Optimization Methods Based on Response Surfaces(by D. R. Jones, one of the authors on the EGO paper) is a nice overview of various optimization methods. It presents each method and gives ways in which it can be fooled into choosing the wrong point as the optimum. A fix is given, which then leads to the next method in the list.


One of the core ideas is regression based on Gaussian processes (kriging). To learn more about Gaussian processes, see www.gaussianprocess.org. The two Jones papers listed above also show how the GP based methods fit in with other basis set methods.

Useful paper for VMC optimization?

I ran across this paper, An informational approach to the global optimization of expensive-to-evaluate functions, and wondered if it might be useful for VMC optimization?

The type of problem being solved seems to fit:

1. Function is expensive to evaluate.
   
2. Function evaluation may be noisy.
   

Many optimization methods estimate an optimal point and evaluate the function at that point. The method described in the paper estimates the uncertainty in the knowledge of the function, and evaluates the function where it will best improve our knowledge of the function.

Some potential issues:

• This method doesn't make use of gradient information, which may make it less competitive than methods that do use gradient information.


• How well does it scale with the number of parameters? The authors present a method to keep the cost under control as the dimension increases, but is the method still effective then?